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DISCUSSION OF AD HOC COMMITTEE TERMS OF REFERENCE 


Overview 

The Sub-Group Panel is charged with the responsibility for 
personnel actions in several areas. This charge is spelled 
out in EXCOM 9030-80. In carrying out these responsibilities 
the members of the Panel are expected to inform themselves 
adequately with respect to the salient characteristics of 
each person considered by the Panel and, further, to apply 
their knowledge of the employees in ways that satisfy the 
criteria of fairness, equity, and considerations of the 
needs of the Agency. In recognition of the fact that these 
criteria are nore adequately met by the use of systematic, 
organized approaches than by informal, impressionistic 
applications of Panel members’ hunches and feelings, the 
Executive Committee has directed that each Career Service 
devise a statement of Panel procedure which will spell out 
in some detail the procedural methods whereby that Panel 
will analyze information about employees and further how the 
results of such analysis will be used to arrive at recommend- 
ations for personnel actions. The Executive Committee is 
specific in stating that each Career Service will design a 
"Work Sheet" to be used in the deliberations of the Panels. 
The specific characteristics for this work sheet are left to 
the discretion of the individual career services in recognition 


of the fact that each has different kinds of problems to 
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address in personnel evaluation. While general performance 
criteria are mandated by the Executive Committee, the 
Specific application and interpretation of these criteria 
are left to the Career Services (subject, of course, to 
appropriate approval). 

The task posed to the committee making the recommendations 
given below was to apply the general mandates of the Executive 
Committee to the RR Career Service. We have ee directed 
to recommend a work sheet. for use by ORD Sub-Group Panels 
and procedures for its use. In doing this we have been 
instructed to take into consideration the unique problems of 
personnel appraisal in the research environment. Further, 
we have been directed to address the process of personnel 
appraisal at all GS levels for which the Career Service Panel 
procedures are appropriate under existing Agency regulations. 
Lastly, we have been mandated to apply, fe the degree possible, 
the shilogepny of "evaluation against objective criteria;" 
this in keeping with management desires in the area of the 
reduction of subjectiveness in personnel evaluation. 

The recommendations Tepresent the result of our attempts 
to reconcile often conflicting (and sometimes mutually 
exclusive) desiderata into a practical, useful, and 
acceptable methodology which, we believe, incorporates the 
letter and spirit of the Executive Committee's mandates. 
Preliminary exposure to both assessors and assessees suggests 


that acceptance will be forthcoming. We hope that ORD 
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management will find these recommendations workable, fair, 
and responsive to the desires of the Executive Committee. 
Desirable Characteristics 

Standardization. Any procedures proposed for use by 
Sub-Group Panels should be such as to make explicit the 
methods by which the decisions of the panel are reached. 
The principal purpose of such standardization is to make, 
insofar as possible, the decisions of any panel independent 
of the particular individuals comprising that panel. In 
other words, standardization would have been achieved when 
it could be demonstrated that several differently constituted 
panels would reach the same conclusions, given the same data 
about the employees to be appraised, It is highly unlikely 
that decisions can be totally independent of panel membership, 
but the criterion of procedural standardization should be a 
goal toward which to strive. | 

Objectivity. The current interest shown by management 
in the philosophy of objectivity in assessment should be 
reflected in any recommendations for panel procedures. The 
key concept is that of verifiability. As applied to the 
area of personnel appraisal, this means (as is stated in 
several personnel management guides) that there should be an 
explicit understanding by both the employee and management 
about exactly what actions, duties, behavior, and outcomes 
are expected. The degree to which these standards are or 
are not met will then form the basis for appraisals and 


subsequently, rewards or adverse action. 
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In the process of appraisal by the Sub-Group Panel, 
this objective approach should be adopted insofar as possible. 
Further, since panel members must make judgments where 
differences are frequently so slight as to make discrimination 
very difficult, panel procedures should, by their nature, 
guide panel members toward objectivity in their decisions, 

Criteria. The criteria by which Sub-Group Panels appraise 
employees have been stated, in the general sense, by 
the Executive Committee. - However, the elaboration of these 
criteria is left to each career service. In this way a 
degree of uniformity will exist across career services, but 
the system in its totality will not be excessively Procrustean. 

‘The characteristics which must be considered by the RR 
Career Service must be concerned with the fact that the 
outcome of research activities is not a "product" in the 
usual sense of the word. Further, it is the nature of the 
research process that it deals, to a great extent, with 
activities whose outcomes are not only initially, undefined 
but often undefinable in principle. It is a truism to state 
that if the answer is known before the project starts there 
is no need to do the research. 

This setting in which research personnel work provides, 
then, the basis for explication of the fundamental appraisal 
criteria as provided by the Executive Committee. These 


criteria should be expanded in such a way as to recognize 
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that research personnel work in domains of somewhat greater 
ambiguity than many other workers. Indeed, one of the 
principal criteria against which they should be judged is 
the facility with which they handle this ambiguity so as to 
reduce it for others. 

Research workers must also maintain unusually high 
level of technical knowledge in a variety of areas and must 
simultaneously function in the administrative world of 
contract management. These considerations, then, should 
guide an attempt to expand the Executive Committee's criteria 
as they apply in the RR Career Service. 

Flexibility. A career Service Panel work sheet (and 
the procedures for its use) must not impose undue restrictions 
on the actions of the Panel. The function of such things as 
work sheets is £6 standardize the process of applying human 
judgment, not to control it. The element of judgment must 
be paramount; the rules and procedures stieuid be enabling 
not constricting. This implies that final outcomes of the 
panels' deliberations should be the direct result of the 
application of judgment. The methods, work sheets, and 
rules should have helped, guided, and generated an audit 
trail leading to the panels' decisions. Such aids must 
never be of such a nature as to generate the decisions 
"automatically" thus relieving the panel members of either 
the ability or the responsibility for making the final 


decisions. 
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Acceptability. Procedural aids in personnel appraisal 
must be acceptable both to the appraisers and the appraisees, 
The absence of the former will result in the circumvention 
of the "aids" because they are not seen as helpful. Acceptance 
by the appraisees is necessary because without it morale 
and, therefore, performance will suffer. This is not to say 
that universal acceptance of any method is necessary (or 
even possible), but substantial acceptance is certainly a 
characteristic which should be high on the list of anyone 
designing personnel appraisal methods or aids. 

Validity. After the management, administrative, and 
humanistic requirements for a performance appraisal aid have 
been met, there still stands the need to make any such 
methodology scientifically and statistically sound. In 
meeting this requirement the designer must take into con- 
sideration what has been demonstrated in research ‘in the field of 
the assessment of Heepie and the appraisal of performance on 
the job. In considering this body of literature the designer 
must take into consideration such things as the reliability 
of the methods proposed. By this is meant the degree to 
which consistency is present; consistency across appraisers 
and across time by the same appraiser. Additionally, attention 
must be paid to validity. That is, "Do the methods generate 
results that coincide with reality?" Obviously, a method 
can have reliability without validity but the reverse is not 


true. Only a knowledge of the field of performance appraisal 
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and the application of this knowledge will result in a 
methodology with both reliability and validity. 

These, then, are the major characteristics whose 
maximization must be attempted if an adequate work sheet 
(with procedures) is to be generated for use by the Career 
Service Panel of the RR Career Service. It is the belief of 
the members of the committee that these matters have received 
adequate and appropriate attention in our efforts to fulfill 
our charge. Our recommendations are given in the attached 


"Recommended Procedures for ORD Sub-Group Panels". 
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